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1. Introduction 


One of the features of modern data is the data dimension d is high and the 
sample size n is relatively low. We call such data HDLSS data. In HDLSS situa¬ 
tions such as d/n —)■ cx), new theories and methodologies are required to develop 
for statistical inference based on the large sample theory. One of the approaches 
is to study geometric representations of HDLSS data and investigate the possi¬ 
bilities to make use of them in HDLSS statistical inference. Hall et al. (2005), 
Ahn et al. (2007), and Yata and Aoshima (2012) found several conspicuous geo¬ 
metric descriptions of HDLSS data when d ^ oo while n is fixed. The HDLSS 
asymptotic studies usually assume either the normality as the population distribu¬ 
tion or a p-mixing condition as the dependency of random variables in a sphered 
data matrix. See Jung and Marron (2009) and Jung et al. (2012). However, Yata 
and Aoshima (2009) developed an HDLSS asymptotic theory without assuming 
those assumptions and showed that the conventional principal component anal¬ 
ysis (PCA) cannot give consistent estimation in the HDLSS context. In order 
to overcome this inconvenience, Yata and Aoshima (2012) provided the noise- 
reduction (NR) methodology that can successfully give consistent estimators of 
both the eigenvalues and eigenvectors together with the principal component (PC) 
scores. Furthermore, Yata and Aoshima (2010, 2013) created the cross-data- 
matrix (CDM) methodology that is a nonparametric method to ensure consistent 
estimation of those quantities. Given this background, Aoshima and Yata (2011, 
2013) developed a variety of inference for HDLSS data such as given-bandwidth 
confidence region, two-sample test, test of equality of two covariance matrices, 
classification, variable selection, regression, pathway analysis and so on along 
with the sample size determination to ensure prespecified accuracy for each infer¬ 
ence. 

In this paper, suppose we have a d x n data matrix, X(d) = [xi^d), •••, Xn{d)], 
where Xj(^d) = ixij{d): •••, Xdj{d)Y'■, j = 1, ...,n, are independent and identically 
distributed (i.i.d.) as a d-dimensional distribution with a mean vector and 
covariance matrix (> O). We assume n > 3. The eigen-decomposition of 

is given by Tid = HdNdH^, where A^; is a diagonal matrix of eigenvalues, 
Ai(d) > • • • > Ad(d)(> 0 ), and Hd = [h^d), •••, ^d(d)] is an orthogonal matrix of 
the corresponding eigenvectors. Let ...,/x^] = HdN]/'^Z(^d)- Then, 

Z(^d)^^ ^dx n sphered data matrix from a distribution with the zero mean and the 
identity covariance matrix. Here, we write Z(^d) = [^i(d); •••; and Zj(^d) = 

(zjK^d), ■■■,Zjn{d)V, j = l,...,d. Note that E(zji(^d)Zj'iid)) = 0 (j 7 ^ f) and 
Nar{zj(^d)) = In, where In is the n-dimensional identity matrix. The i-th true PC 
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score of is given by - /x^) = (hereafter ealled Sij(rf)). 

Note that Nds{sij(d)) = \{d) for all i,j. Hereafter, the subseript d will be omitted 
for the sake of simplieity when it does not cause any confusion. We assume that 
Ai has multiplieity one in the sense that liminf^^oo A 1 /A 2 > 1. Also, we assume 
that limsupi?( 2 :A) < cxd for alH, j and P(limd^.oo || 2 i|| 7 ^ 0) = 1. Note 
that if X is Gaussian, ZijS are i.i.d. as the standard normal distribution, A^(0,1). 
As neeessary, we eonsider the following assumption for the normalized first PC 
seores, zij (= 

(A-i) zij, j = 1 ,n, are i.i.d. as A^( 0 , 1 ). 

Note that P(lim(i_j.oo ||^i|| 7 ^ 0) = 1 under (A-i). Let us write the sample eo- 
variance matrix as S = {n — 1)“^(X — X){X — X)'^ = {n — 1)“^ ~ 

x){xj — xY, where X = [x,...,x] and x = YTj=i Xj/n. Then, we define the 
n X n dual sample covariance matrix hy So = (n — 1)“^(X — X)^(X — X). 
Let Ai > ■ ■ ■ > A„_i > 0 be the eigenvalues of Sn- Let us write the eigen- 
deeomposition of Sd Sd = ^ where iij = (fiji,..., de¬ 

notes a unit eigenvector corresponding to \j. Note that S and So share non-zero 
eigenvalues. 

In this paper, we study asymptotic properties of the first principal component 
in the HDLSS eontext and apply them to equality tests of eovarianee matriees for 
high-dimensional data sets. We eonsider HDLSS asymptotic theories as d ^ 00 
for both the eases when n is fixed and n ^ 00 . In Seetion 2, we introduce an 
eigenvalue estimator by the NR methodology and provide asymptotic distributions 
of the largest eigenvalue in the HDLSS eontext. We construet a eonfidenee inter¬ 
val of the first eontribution ratio. In Seetion 3, we give asymptotie properties both 
for the first PC direction and PC score as well. In Seetion 4, we apply the findings 
to equality tests of two eovarianee matriees in the HDLSS context. Finally, in See¬ 
tion 5, we provide numerieal results and diseussions about the performanees both 
on the estimates of the first PC and the equality tests of two eovarianee matrices. 

2. Largest eigenvalue and its contribution rate 

In this section, we give asymptotie distributions of the largest eigenvalue and 
eonstruet a eonfidenee interval of the first eontribution rate. 

2.1. Asymptotic distributions of the largest eigenvalue 

Let 5i = tr(S^) — = J2t=i+i = 1, ...,d — 1. We eonsider the 

following assumptions for the largest eigenvalue: 


3 



6 S 

(A-ii) = o(l) as d —>■ oo when n is fixed; = o(l) as d —)■ oo for some 

Ai 


fixed u (< d) when n ^ oo. 


(A-iii) 




is fixed or n —)• cx). 


nXl 


= o(l) as d ^ oo either when n 


Note that (A-iii) holds when X is Gaussian and (A-ii) is met. Let Zoj = Zj — 
j = where Zj = ri-^ Ylk=i^jk- Let n = tr(S) - Ai = 

Ylt =2 Then, we have the following result. 

Proposition 2.1. Under (A-ii) and (A-iii), it holds that 

Y -\\Zoi/s/n-l\\^ - ^ = Op(l) 

Ai Ai(n-l) 

as d ^ oo either when n is fixed or n ^ oo. 


Remark 2.1. Jung et al. (2012) gave a result similar to Proposition 2.1 when X 
is Gaussian, /x = 0 and n is fixed. 

It holds that E{\\zoi/s/n — l|p) = 1 and \\zoi/s/n — l|p = 1 -f Op(l) as 
n —)■ oo. If K/(n\i) = o(l) as d —)■ cxD and n —)■ cx), Ai is a consistent es¬ 
timator of Al. When n is fixed, the condition ‘k/Ai = o(l)’ is equivalent to 
‘Ai/tr(I]) = 1 -f o(l)’ in which the contribution ratio of the first principal compo¬ 
nent is asymptotically 1. In that sense, ‘k/Ai = o(l)’ is quite strict condition in 
real high-dimensional data analyses. Hereafter, we assume liminf^^oo > 0. 

Yata and Aoshima (2012) proposed a method for eigenvalue estimation called 
the noise-reduction (NR) methodology that was brought by a geometric represen¬ 
tation of Sd- If one applies the NR methodology to the present case, AjS are 
estimated by 


A* 



trjSp) - Yl]=i Aj 

n — 1 — i 


(i = l,...,n-2). 


( 2 . 1 ) 


Note that A* > 0 w.p.l for i = 1, —2. Also, note that the second term in (12.11) 

with i = 1 is an estimator of fi:/(n — 1). See Lemma 2.1 in Section 2.2 for the 
details. Yata and Aoshima (2012, 2013) showed that Aj has several consistency 
properties when d —)■ oo and n —)■ oo. On the other hand, Ishii et al. (2014) gave 
asymptotic properties of Ai when d —)■ oo while n is fixed. The following theorem 
summarizes their findings: 
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Theorem 2.1. Under (A-ii) and (A-iii), it holds that as d ^ oo 


Ai I I \Zoi/\/n — 1| p + Op(l) when n is fixed, 

Ai [ 1 + c'p(l) when n —)■ cxo. 

Under (A-i) to (A-iii), it holds that as d ^ oo 


(^- 1)^ ^ xl-i 


when n is fixed, 
when n —)■ oo. 


Here, “ ^ ” denotes the convergence in distribution and Xn-i denotes a random 
variable distributed as distribution with n — 1 degrees of freedom. 


2.2. Confidence interval of the first contribution ratio 

We consider a confidence interval for the contribution ratio of the first princi¬ 
pal component. Let a and b be constants satisfying P{a < xl-i <h) = l-a, 
where a G (0,1). Then, from Theorem 2.1, under (A-i) to (A-iii), it holds that 


P( 


Ai 


In 


1)A 


n 


l)Ai 


•tr(S) -bn + (n — l)Ai ^ an + (n — 1)A 


( 2 . 2 ) 


as d —)■ cxo when n is fixed. We need to estimate n in (12.21) . Here, we give a 
consistent estimator of nhy k = (n — l)(tr(5£)) — Xi)/{n — 2) = tfiSn) — Ai. 
Then, we have the following results. 


Lemma 2.1. Under (A-ii) and (A-iii), it holds that 


^ 

— = 1 -f Op(l) and — 

K Ai 

as d ^ oo either when n is fixed orn —)■ oo. 


n 

aI 


+ Op(l) 


Theorem 2.2. Under (A-i) to (A-iii), it holds that 


P 


Ai 


[n 


l)Ai 


[n 


1 )^ 


Ar(T,) _|_ (t^i _ x)_\j^ jf- (fi — x)_\^ 

as d ^ oo when n is fixed. 


= l-a + o(l) (2.3) 
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Remark 2.2. From Theorem 2.1 and Lemma 2.1, under (A-ii) and (A-iii), it holds 
that tr(5D)/tr(S) = {k + Ai)/tr(S) = 1 + Op(l) as d —)■ oo and n —)■ oo. We 


have that 


Ai 

tr(5,,) 


Ai 

tr(S) 


{1 + Op(l)}. 


Remark 2.3. The eonstants (a, b) should be ehosen for (12.31) to have the minimum 
length. If Ai/k = o(l), the length of the eonfidenee interval beeomes elose to 
{{n — l)Ai/K}(l/a — 1/6) under (A-ii) and (A-iii) when d ^ oo and n is fixed. 
Thus, we reeommend to ehoose eonstants (a, b) sueh that 


argmin(l/a — 1/6) subject to Gn-i{b) — G'„_i(a) = 1 — a, 

a,b 

where G„_i(-) denotes the c.d.f. of xi-i- 

Let us construct a confidence interval for the contribution ratio of the first 
principal component. We used gene expression data by Armstrong et al. (2002) in 
which the data set consists of 12582 (= d) genes. The data set has three leukemia 
subtypes: 24 samples from acute lymphoblastic leukemia (ALL), 20 samples from 
mixed-lineage leukemia (MLL), and 28 samples from acute myeloid leukemia 
(AML). We standardized each sample so as to have the unit variance. Then, it 
holds tr(5) (= tr(5D)) = d, so that Xi + k = d. From Theorem 2.2, we con¬ 
structed a 95% confidence interval of the first contribution rate for each data set 
by choosing (a, 6) as in RemarkThe results are summarized in Table 1. 

Table 1. The 95% confidence interval (Cl) of the first contribution ratio, together 
with Al and k, for Armstrong et al. (2002)’s data sets having d = 12582. 



Cl 

Al 

K 

ALL 

[n = 24) 

[0.0557,0.1663] 

1256 

11326 

MLL 

(n = 20) 

[0.1201,0.3458] 

2717 

9865 

AML 

(n = 28) 

[0.0706,0.1884] 

1501 

11081 


3. First PC direction and PC score 

In this section, we give asymptotic properties of the first PC direction and PC 
score in the HDLSS context. 
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3.1. Asymptotic properties of the first PC direction 

Let H = [^1, hf\, where H is a d x d orthogonal matrix of the sample 
^ r - - - ^ ^ 

eigenvectors such that H SH = A having A = diag(Ai,A^). We assume 

hjh, > 0 w.p. 1 for all i without loss of generality. Note that hi can be calculated 

by = {(n — — X)ui. First, we have the following result. 

Lemma 3.1. Under (A-ii) and (A-iii), it holds that 

hlh^ - (l+ 2 112 ) ^ =Op(l) 

as d ^ 00 either when n is fixed or n ^ 00 . 

If K/{n\i) = 0(1) as d —)■ cxD and n —)■ cx), is a consistent estimator of 

hi in the sense that h^hi = 1 + Op(l). When n is fixed, hi is not a consistent 
estimator because limd-j.oo «/Ai > 0. In order to overcome this inconvenience, 
we consider applying the NR methodology to the PC direction vector. Let hi = 
{{n — — X)ui. From Lemma 3.1, we have the following result. 

Theorem 3.1. Under (A-ii) and (A-iii), it holds that 

~ T 

h^ fi-i = 1 + Op(l) 

as d ^ 00 either when n is fixed or n ^ 00 . 

Note that = Ai/Ai > 1 w.p.l. We emphasize that hi is a consistent 

estimator of hi in the sense of the inner product even when n is fixed though hi 
is not a unit vector. We give an application of hi in Section 4. 

3.2. Asymptotic properties of the first PC score 

Let Zoij = Zij — Zi for all i,j. First, we have the following result. 

Lemma 3.2. Under (A-ii) and (A-iii), it holds that 

uij = Zoij/\\Zoi\ \ + Op(l) forj = 1, ...,n 
as d ^ 00 when n is fixed. 

Remark 3.1. By using Lemma 3.2 and the test of normality such as Jarque-Bera 
test, one can check whether (A-i) holds or not. 
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By applying the NR methodology to the first PC score, we obtain an estimate 

by sij = \J{n — l)XiUij, j = 1, n. A sample mean squared error of the first 
PC score is given by MSE(si) = n~^ ~ Then, from Theorem 2.1 

and Lemma 3.2, we have the following result. 

Theorem 3.2. Under (A-ii) and (A-iii), it holds that 

1 

- Sy) = -Zx + Op(l) forj = 1, ...,n 

V Ai 

as d ^ oo when n is fixed. Under (A-i) to (A-iii), it holds that 





Ai 


(sij — sij) ^ N(Q, 1) for j = 1, ...,n; and 


MSEisi) 2 
n - 

Ai 


as d ^ oo when n is fixed. 


Remark 3.2. The conventional estimator of the first PC score is given by sij = 

\J(n — j = 1,..., n. From Theorems 8.1 and 8.2 in Yata and Aoshima 

(2013), under (A-ii) and (A-iii), it holds that as d ^ oo and n —)■ cxd 


MSE(si) 

Ai 


Op(l) ifn/(nXi) 


o{l), 


and 


MSE(si) 

Ai 


Op(l). 


4. Equality tests of two covariance matrices 

In this section, we consider the test of equality of two covariance matrices in 
the HDLSS context. Even though there are a variety of tests to deal with covari¬ 
ance matrices when d ^ oo and n —)■ cx), there seem to be no tests available in 
the HDLSS context such as d —)■ cxd while n is fixed. Suppose we have two inde¬ 
pendent d X Ui data matrices, Xi = [a3i(i),..., * = 1,2, where j = 

1 , ...,nj, are i.i.d. as a d-dimensional distribution, tt^, having a mean vector 
and covariance matrix Ej (> O). We assume > 3, f = 1,2. The eigen- 
decomposition of Sj is given by Ej = HiAiHj, where Aj = diag(Ai(i),..., \d{i)) 
having Ai(j) > • • • > 0) and Hi = ..., h^^i)] is an orthogonal matrix 

of the corresponding eigenvectors. 
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4.1. Equality test using the largest eigenvalues 

We consider the following test for the largest eigenvalues: 


Ho ■ Ai(i) = Ai( 2 ) vs. Ha : Ai(i) 7 ^ Ai(2) (or Hb : Ai(i) < Ai(2)). (4.1) 

Let Ai(i) be the estimate of Ai(i) by the NR methodology as in (12.11) for tt*. Let 
z/i = ni — 1 and 1^2 = n 2 — 1. From Theorem 2.1, we have the following result. 

Corollary 4.1. Under (A-i) to (A-iii)for each TTj, it holds that 


Ai(i)/Ai(i) 

Ai(2)/Ai(2) 




as d ^ 00 when riiS are fixed, where denotes a random variable distributed 
as F distribution with degrees of freedom, ui and 02 - 

Let Fi = Ai(i)/Ai( 2 )- From Corollary 4.1, we test (14.11) for given a G (0,1/2) 
by 


accepting Ha F^ ^ [{F^^^^fa/2)} \ F^^^^^{a/2)] 
or accepting Hb Fi < 


(4.2) 

(4.3) 


where denotes the upper a% point of F distribution with degrees of 

freedom, ui and z/ 2 . Then, under (A-i) to (A-iii) for each vr*, it holds that 


size = a + o(l) 


as d —)■ 00 when n^s are fixed. 

Now, we check the performance of the test by (14.21) or (14.31) . We also consider 
a test by the conventional estimator, Ai(j). Let k* = tr(Si) — Ai(j) = 
for i = 1,2. From Proposition 2.1, if Ki/X^i) = o(l), i = 1,2, under (A-i) to 
(A-iii) for each vr^ it holds that 


Ai(i)/Ai(i) 

Ai(2)/Ai(2) 


U1,U2 


asd ^ 00 when rijS are fixed. As mentioned in Section 2, the condition ‘Kj/Ai(j) = 
0 ( 1 ) for i = 1, 2’ is quite strict in real high-dimensional data analyses. Hereafter, 
we assume liminf^^oo «^i/Ai(i) > 0 for i = 1,2. We analyzed the same gene 
expression data as in Table 1. We set a = 0.05. We considered two cases: (I) 
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TTi : ALL (ni = 24) and 112 : MLL (n 2 = 20), and (II) tti : AML (ni = 28) and 
7^2 ; MLL {712 = 20). As for F[ = Ai(i)/Ai( 2 ), we considered (14.21) and (14.31) by 
replacing Fi with F[. The results are summarized in Table 2. We observed from 
Table 2 that only Hb for (I) was accepted by Fi, namely, only Fi for (I) rejected 
Ho vs. Hb. One should note that the condition ‘Ki/Ai(j) = o(l) for i = 1, 2’ does 
not hold both for (I) and (II) as observed in Table 1. 

Table 2. Tests of Ho : Ai(i) = Ai( 2 ) vs. Ha : Ai(i) ^ Ai( 2 ) or Hb : Ai(i) < Ai( 2 ) 
with size 0.05 for Armstrong et al. (2002)’s data sets having d = 12582. 



HahyF^ 

HahyF[ 

Hb by Fi 

Hb by F[ 

(I) TTi: ALL, tt2: MLL 

(II) TTi: AML, 712-. MLL 

Reject 

Reject 

Reject 

Reject 

Accept 

Reject 

Reject 

Reject 


4.2. Equality test using the largest eigenvalues and their PC directions 

We consider the following test using the largest eigenvalues and their PC di¬ 
rections: 


Hq ■ (Ai(i), b,i(i)) — (Ai( 2 ), b.i( 2 )) vs. Ha : (Ai(i), b.i(i)) 7 ^ (Ai( 2 ), b,i( 2 )). 

(4.4) 

Let hi(i') be the estimator of the first PC direction for vTj by the NR methodology 
given in Section 3.1. We assume > 0 w.p.l for i = 1,2, without loss of 

generality. Here, we have the following result. 


Lemma 4.1. Under (A-ii) and (A-iii)for each tt^, it holds that 

~ ~ rp 

^1(1)^1(2) = ^1{1)^1(2) + Op(l) 
as d ^ 00 either when rii is fixed or rii 00 for i = 1,2. 

Let h = 1/2 + |/i]^(-]^)fi,i(2)|“^/2. Note that h > 1. Then, from 

Lemma 4.1, we give a test statistic for (14.41) as follows: 


F2 = 

A 


1 ( 2 ) 

where 

h if Ai(i) > Ai( 2 ), 
li~^ otherwise. 

From Lemma 4.1, we have the following result. 


= 
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Theorem 4.1. Under (A-i) to (A-iii)for each tt^, it holds that 


F2 


under Hq 


as d ^ 00 when riiS are fixed. 

From Theorem 4.1, we eonsider testing (14.41) by (14.21) with F 2 instead of Fi. 
Then, the size beeomes elose to a as d inereases. For the same gene expression 
data sets as in Section 4.1, we tested (14.41) with a = 0.05 for the cases of (I) and 
(II). We observed that only Ha for (II) was accepted by F 2 , namely, only F 2 for 
(II) rejected Hq vs. Ha in (14.41) . 

4.3. Equality test of the covariance matrices 

We consider the following test for the covariance matrices: 


Ho '■ "Fii — S 2 vs. Ha '. "Si f S 2 . 


(4.5) 


When d ^ 00 and n^s are fixed, one cannot estimate Xj{i)S and h-j(i)S for j = 
2, ...,d. Instead, we consider estimating k^s. Let Sd{i) be the dual sample co- 
variance matrix for vTj. We estimate Ki by Ri = tr{S£){i)) — X^i) for i = 1,2. 
From Lemma 2.1, under (A-ii) and (A-iii) for each tt^, RiS are consistent estima¬ 
tors of KjS in the sense that Ri /= 1 -f Op(l) as d ^ 00 when rZjS are fixed. Let 
7 = max{Ri/R2, ^ 2 /^ 1 }. Now, we give a test statistic for (14.51) as follows: 



where 



7 if Ai(i) > Ai(2), 
7 “^ otherwise. 


Then, we have the following result. 


Theorem 4.2. Under (A-i) to (A-iii) for each tt^, it holds that 


F3 F, 


under Hq 


as d ^ 00 when riiS are fixed. 
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From Theorem 4.2, we eonsider testing (14.51) by (14.21) with F 3 instead of Fi. 
Then, the size beeomes close to a as d increases. For the same gene expression 
data sets as in Section 4.1, we tested (14.51) with a = 0.05 for the cases of (I) and 
(II). We compared the performance of F 3 with two other test statistics: Q 2 and T| 
by Srivastava and Yanagihara (2010). The results are summarized in Table 3. We 
observed that Ha was accepted by F 3 both for (I) and (II), namely, F 3 rejected Hq 
vs. Ha in (14.51) for both the cases. On the other hand, and T| did not work for 
these data sets. It should be noted that and T| require to meet the conditions 
that 0 < limd_ 5 .oo tr(S*)/(i < 00 (i = 1, ...,4) and = o(l). As observed in 

Table 1, the conditions seem not to hold for these data sets with d = 12582 and 
n < 28. Hence, there is no theoretical guarantee for the results by Q 2 and T^. 

Table 3. Tests of Hq : Jli = S 2 vs. Ha ■ 'Si ^ S 2 with size 0.05 for Armstrong 
et al. (2002)’s data sets having d = 12582. 



Ha by F 3 

Ha by Ql 

Ha by Ti 

(I) TTi: ALL, 7r2: MLL 

(II) TTi: AML, 7r2: MLL 

Accept 

Accept 

Reject 

Reject 

Reject 

Reject 


5. Numerical results and discussions 

5.1. Comparisons of the estimates on the first PC 

In this section, we compared the performance of Ai, hi and Sij with their con¬ 
ventional counterparts by Monte Carlo simulations. We set d = 2*^, k = 3, ..., 11 
and n = 10. We considered two cases for AjS: (a) Aj = ^ = 1, ...,d and 

(b) Aj = i = l,...,d. Note that Ai = d for (a) and Ai = d^/^ for 

(b). Also, note that (A-ii) holds both for (a) and (b). Let d* = where 

[x] denotes the smallest integer > x. We considered a non-Gaussian distribu¬ 
tion as follows: {zij,..., Zd-d,jV, j = l,...,n, are i.i.d. as Nd-dfiO,Id-dJ 
and {zd-d,+ij, ■■■, Zdj)'^, j = l,...,n, are i.i.d. as the d*-variate f-distribution, 
fd*(0, Id,, 10) with mean zero, covariance matrix Id, and degrees of freedom 10, 
where {zij ,..., Zd-d,jY' and {zd-d,+ij, Zdff^ are independent for each j. Note 
that (A-i) and (A-iii) hold both for (a) and (b) from the fact that J2ts>2 ^r^sE{{z‘^j.— 

i)(4 -1)} = 2 Ett A? + o{El>d-d,^i KXs) = o{xi). 

The findings were obtained by averaging the outcomes from 2000 (= R, say) 
replications. Under a fixed scenario, suppose that the r-th replication ends with 
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Figure 1 . The valuesofA:Ai/Ai and B : A i / A i are denoted by the dashed lines for 
(a) and by the dotted lines for (b) in the left panel. The values of A: var(Ai/Ai) and 
B: var(Ai/Ai) are denoted by the dashed lines for (a) and by the dotted lines for (b) 
in the left panel. The asymptotie varianee of Ai/Ai was given by Var{x^_i/(n — 
1)} = 0.222 and denoted by the solid line in the left panel. 

estimates, (Ai^, hir, MSE(si)r) and (Ai^, MSE(si)r) (r = 1, Eet us 

simply write Ai = R~^ ^ir and Ai = i? ^ Ylr=i We also eonsidered 
the Monte Carlo variability by var(Ai/Ai) = {R — 1)“^ ~ 

var(Ai/Ai) = {R — 1)“^ Eigure 1 shows the behaviors of 

(Ai/Ai, Ai/Ai) in the left panel and (var(Ai/Ai), var(Ai/Ai)) in the right panel 
for (a) and (b). We gave the asymptotie varianee of Ai/Ai by Nds{x\-i/{n — 
1)} = 0.222 from Theorem 2.1 and showed it by the solid line in the right panel. 
We observed that the sample mean and varianee of Ai/Ai beeome elose to those 
asymptotie values as d inereases. 

~T ''T ~T 

Similarly, we plotted {h^ hi, hR and (var(h,^ hi), \ar{h^ hi)) in Eigure 
2 and (MSE(si)/Ai, MSE(si)/Ai) and (var(MSE(si)/Ai), var(MSE(si)/Ai)) in 
Eigure 3. Erom Theorem 3.2, we gave the asymptotie mean of MSE(si)/Ai by 
E{xi/n) =0.1 and showed it by the solid line in the left panel of Eigure 3. We 
also gave the asymptotie varianee of MSE(si)/Ai by Var(x^/n) = 0.02 in the 
right panel of Eigure 3. Throughout, the estimators by the NR method gave good 
performanees both for (a) and (b) when d is large. However, the eonventional 
estimators gave poor performanees espeeially for (b). This is probably beeause 
the bias of the eonventional estimators, k/ (nAi), is large for (b) eompared to (a). 
See Proposition 2.1 for the details. 
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- T ~ T 

Figure 2. The values of A: hi and B: hihi are denoted by the dashed lines for 

- T 

(a) and by the dotted lines for (b) in the left panel. The values of A: var(^^ hi) 

~ T 

and B: var(h,^ hi) are denoted by the dashed lines for (a) and by the dotted lines 
for (b) in the right panel. 



Figure 3. The values of A: MSE(si)/Ai and B: MSE(si)/Ai are denoted by the 
dashed lines for (a) and by the dotted lines for (b) in the left panel. The values 
of A: var(MSE(si)/Ai) and B: var(MSE(si)/Ai) are denoted by the dashed lines 
for (a) and by the dotted lines for (b) in the right panel. The asymptotie mean and 
varianee of MSE(si)/Ai were given by E{x\/n) = 0.1 and Var(x?/'n) = 0.02 
and denoted by the solid lines in both the panels. 
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5.2. Equality tests of two covariance matrices 

We used eomputer simulations to study the performanee of the test proeedures 
by Fi for ( 14 . 11 ) . F 2 for ( 14 . 41 ) and F 3 for ( 14 . 51 ) . We set a = 0.05. Independent 
pseudo-random normal observations were generated from TTj : Nd{0, = 1,2. 

We set (rii, nf) = (10, 20). We eonsidered the eases: d = 2^, k = 3, ..., 11, and 



where Ok,i is the fc x Z zero matrix, Si(i) = diag((i^/'^, and Si(2) = (O.Sl^ *1). 
When eonsidered the alternative hypotheses, we set 



and ^2(2) = 1.5(0.31* *1). Note that Ai(2)/Ai(i) = 3, K 2 /K 1 = 1.5, = 

(1, 0,...., 0)^ and/ii(2) = (1/3, \/8/3, 0...., 0)^, so that h,^^)h,i(2) = 1/3. Also, 
note that (A-i) to (A-iii) hold for eaehTrj. Let h = {\h^^~^hi(^ 2 )\ + ^/\h^i)hi( 2 )\)/‘^ 
and 7 = max{fi:i/fi:2, K2 /«i}- From Lemmas 2.1 and 4.1, it holds that h = 
h + Op(l) and 7 = 7 + Op(l). Thus, from Corollary 4.1, Theorems 4.1 and 4.2, 
we obtained the asymptotie powers of Fi, F 2 and F3 with (h*, 7*) = (h“^, 7“^) 
as follows: 


Power(Fi) = P{(Ai(i)/Ai(2))/ ^ [{F^^^^^{a/2)}-\ F^^^^^{a/2)]} = 0.39, 
Power(F2) = P{h"^(Ai(i)/Ai(2))/ ^ [{F^^^ui{a/2)}-^, F^^^^^{a/2)]} = 0.726 
and Power(F3) = P{7"^h“^(Ai(i)/Ai(2))/ ^ [{F^^^^^{a/2)}-\ F^^^^^{a/2)]} 


0.908, 


where / denotes a random variable distributed as P distribution with degrees of 
freedom, ui and 02 . Note that Power(P2) and Power(P3) give lower bounds of the 
asymptotie powers when /i* = h~^ and 7* = 7“^. 

In Figure 4, we summarized the findings obtained by averaging the outeomes 
from 4000 (= P, say) replieations. Here, the first 2000 replieations were gen¬ 
erated by setting S2 = Si as in (15.11) and the last 2000 replieations were gen¬ 
erated by setting S2 as in (15.21) . Let Pj^ {i = 1,2,3) be the rth observation 
of Fi for r = 1,...,4000. We defined P^ = 1 (or 0) when Hq was falsely 
rejected (or not) for r = 1 , ..., 2000 , and Ha was falsely rejected (or not) for 
r = 2001, ...,4000. We defined a = (P/2)“^ Zlfii to estimate the size and 
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Size 


Power 




Powers of Fi, F2 and F3 


log,cf 


Figure 4. The values of a are denoted by the dashed lines in the left panel and the 
values of 1 — /9 are denoted by the dashed lines in the right panel for Fi, F 2 and 
F 3 . The asymptotie powers were given by Power(Fi) = 0.39, Power(Fi) = 0.726 
and Power(F 3 ) = 0.908 whieh were denoted by the solid lines in the right panel. 


1 - ,3 = 1 - (fi/2)-‘ Ef.„, 2+1 Pr to estimate the power. Their standard devi¬ 
ations are less than 0.011. Throughout, the tests gave adequate performanees for 
the high-dimensional eases. 


Appendix A. 

Throughout, let = /„ — where In = (1,...,!)^. Let = 

(ei,..., be an arbitrary (random) n-veetor sueh that | |e„| | = 1 and = 0 . 


Proof of Proposition 2.1. We assume p = 0 without loss of generality. We write 
that X^X = for = 1 when n is fixed, and 

for some fixed f*(> 1) when n ^ 00 . Here, by using Markov’s inequality, for 
any r > 0, under (A-ii) and (A-iii), we have that 


j = l S = J* + 1 


>^siZsj - 1)\2 


nXi 




> T> < -- -3- > 0 


rnXl 


E ( E 


j¥=j' S=i» + 1 


XgZsjZgjf 

nXi 


> r ^ < 


rA? 


^ 0 
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as (i —)• oo either when n is fixed or n —)■ oo. Note that YTj=i ^ 
YTj^f CjCj' ^ 1- Then, under (A-ii) and (A-iii), we have that 


^^2 ^ A.(4-l) 


j=l s=i*+l 


nXi 


< 


71/ ^ 2 ^ ^ 


i=i 

= Op(l) and 


E( E 

j = l S=2* + l 


n d > 

E V ^ ^s^sj^sj' 


71 1 /o 't Ci 

^{E»? 4 } {E( E 




= Op(l) 


i4i' s=**+i 


nAi 




as d —)• oo either when n is fixed orn ^ oo. Thus, we elaim that 


XX 

(n - l)Ai 


6n — 


E:=i 

(n — l)Ai ' {n — l)Ai 


+ 


+ Op(l) 


(A.l) 


from the faet that Es=j*+i A^/(n,Ai) = n/ (nAi) + o(l) when n —>■ oo. Note that 
and for all s. Also, note that {Zos/n^^'^)^{Zos'= 

Op(l) for s 7 ^ s' as n —)■ oo from the faet that E{{z'^^Zos//ny} = o(l) as n —)■ oo. 
Then, by noting that P(limd_>.oo | l^oil I 7^ 0) = liminf^^oo A 1 /A 2 > 1 and 
z'^^ln = 0, it holds that 


tEEi ^ sZsZ '^ 
max < ei ———-e,. 


(n - l)Ai 


'EEiA. 




= max , ^ 

Gn I (n — l)Ai 

= \\Zoi/\/n - 1||^ + Op(l) 


(A.2) 


as d —)■ 00 either when n is fixed or n —)■ 00 . Note that = 0 and Pn = 
when Sd O. Then, from (lA.ll) . (IA.2I) and PnX^XP„/(n — 1) = Sd, under 
(A-ii) and (A-iii), we have that 

= illill = W^oiNn - 1||^ -f - --f Op(l) (A.3) 

Ai (n — l)Ai [n — l)Ai 

as d —)■ 00 either when n is fixed or n —)■ 00 . It eoneludes the result. □ 


Proof of Lemma 2.1. By using Markov’s inequality, for any r > 0, under (A-ii) 
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and (A-iii), we have that 


p{ 

s=2 


(n-l)}x2 


(n - l)Ai 


> r 


= O 


s=2 

d 


A,{(n - 1) - 1)/^ - ZskZsk' In }2 

(n - l)Ai 

E.%>2 KKE{{zl,-l){zl,-l)} 

n\\ 


> T 


+ 0}5i/{n\i)^} —)■ 0 


as d —)■ cxD either when n is fixed or n —)■ cxd. Thus it holds that tr(5£))/Ai = 
k/\ i + \\Zoi/y/n — l|p + Op(l) from the faet that tr(5D) = Ai||zoi|P/(^ — 1) + 
Ylt= 2 ^s\\Zos\\'^/{n — 1). Then, from Proposition 2.1 and liminf^^oo > 0, 
we ean elaim the results. □ 


Proof of Theorem 2.1. When n —)■ 00, we ean elaim the results from Theorems 
4.1, 4.2 and Corollary 4.1 in Yata and Aoshima (2013). When n is fixed, by 
eombining Proposition 2.1 with Lemma 2.1, we ean elaim the results beeause 
ll^oilP = EEi ^ik ~ is distributed as xh-i if zij, j = 1,..., k, are i.i.d. as 
iV(0,l). □ 


Proof of Theorem 2.2. From Theorem 2.1 and Lemma 2.1, under (A-i) to (A-iii), 
it holds that 


P 


Ai 


tr(S) 


(n - l)Ai 


{n - l)Ai 


bK, + {n — l)Ai QK, + (n — l)Ai 


= P 


= P 


{n - l)Ai 
bk + {n — l)Ai 
ak 


< 


< 


(n 


l)Ai 


(n - l)Ai 
= 1 — a + 0(1) 


K 

< — < 
" Ai - 


Ai 

tr(S) [ji — x)_\^ 

bk 


{n - l)Ai 


= P{ a < (n 



as d —)• 00 when n is fixed. It eoneludes the result. 


□ 


Proofs of Lemmas 3.1 and 3.2. We note that ||Zoi|p/n = 1 + Op(l) as n —)■ cxd. 
From (IA.3I) . under (A-ii) and (A-iii), we have that 

f^r-^oi/ll-^oill = 1 + c’p(l) (A.4) 
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as (i —)■ oo either when n is fixed orn —)■ cx), so that 2oi = I l^oi 11 + 

Thus, we ean elaim the result of Lemma 3.2. On the other hand, with the help of 
Proposition 2.1, under (A-ii) and (A-iii), it holds that from (IA.4I) 


^ hl{X - X)ui ^ ^ ll^oill + Opin^^"^) 

{(n —l)Ai}^A ji /2 {|l^oilP + «^/Ai + Op(n)}^A 

1 , 

“ {1 + /^/(Ai||;^„i||2)}V2 + 

as d —)■ oo either when n is fixed or n —>■ cx). It eoneludes the result of Lemma 
3.1. □ 


Proof of Theorem 3.1. With the help of Theorem 2.1, under (A-ii) and (A-iii), we 
have that from (IA.4I) 


- _ hl(X-X)u. 
{(n-l)Ai}V2 


ll^oill + Op{rf/^) 

{\M\^ + Op{n)y/^ 


1 + Op{l) 


as d —)• oo either when n is fixed or n ^ oo. It eoneludes the result. 


□ 


Proof of Theorem 3.2. By eombing Theorem 2.1 with Lemma 3.2, under (A-ii) 
and (A-iii), we have that 


^ij /'s/Ai Mij {tl 1)Ai/Ai Kijll■2'oi 11 T ®p(l) ^oij T Op(l) 

as d —)■ oo when n is fixed. By noting that Zoij = zij — zi and zi is distributed as 
A^(0,1/n) under (A-i), we have the results. □ 

Proof of Corollary 4.1. From Theorem 2.1, the result is obtained straightfor¬ 
wardly. □ 

Proof of Lemma 4.1. Let Zi = [2i(j),..., be a sphered data matrix of vr* 

for i = 1, 2, where = {zj^n ,..., Zjni(i)Y■ We assume ^ without 

loss of generality. Let 13st = {Xs(\)Xt( 2 )Y^‘^h^s(i)^t{ 2 ) for all s, t. Let be a fixed 

eonstant sueh that J2t=u+i ^\j)l^lij) ^ '^(^) as d —)■ cx) for j = 1,2. Note that 
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i* exists under (A-ii) for eaeh vTj. We write that 

d 

XiX2= ^ ^ /3stZs(i)Zj(-2) + ^ ^ /^stZs(l)Zt(2) 

S,£<2* S,t>2* + 1 

d iir iir d 

S=2* + l t=l S = 1 t=2* + l 


d ii, 

^{( i: E Pst^sj(l)^tj' {2^ ^ 

s=ii, + l t=l 

d 2* 

^^s{l)h^s(l) ^t{2)ht{2)h^(2^ < AAi^+l(l)Ai(2) 

t=l 


= tr( ^ A,(i) 

s=i* + l 


'. Also, note that 

d 

^ 'y ^ f3stzsj(i)Ztj' 

s,£>2*+1 


forallj,j' 

d ^ d d 

i'(2)) } = ^ \t(2)ht(2)hj(^2)) 

S=2* + l t=2* + l 

c/ 1 

s ( E E ^?(.)) 

s=i*+l t=i*+l 

for all j,j'. Then, by using Markov’s inequality, for any r > 0, under (A-ii) for 
eaeh vr,, we have that 

n\ n2 d ii, n r. 

'f5SiE,Ss&S5F) 

ni 712 j* d o n 

either when n, is fixed orn* —)■ oo for i = 1, 2. Henee, similar to (lA.ll) . 

It 

(i/iI/2Ai(i)Ai(2))^/^ (i/iI/2Ai(i)Ai(2))^/^ 


as d —)■ CX3 . 

it holds that 
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Note that e^,P„. = e^, and = ^oi(i) for i = 1,2, where Zoi^i) = 

zi[i) -_(£i(i), and note that = 

{Xi - Xi) fori = 1 , 2 , where Xi = [ij,...,®*] and Xi = Let 

'Ui(i) be the first (unit) eigenveetor of {Xi — Xj)^(Xj — Xi) for i = 1, 2. Note 
thatn^-^P^, = when (Xj —Xj)^(Xj —Xj) 7 ^ Ofori = 1,2. Then, under 
(A-ii) for eaeh tt*, we have that 

~ ~ X 2 )'Ul{ 2 ) 

(i^l^ 2 Al(l)Ai( 2 ))V 2 - (i^li^ 2 Al(l)Ai( 2 ))V 2 ^ 

_ (A.5) 

as d —)■ cx) either when rzj is fixed or rzj —)■ 00 for i = 1,2. Note that hi(j) = 
{z2iAi(i)}“L2(Xi - Xi)fii(i) for i = 1, 2. Also, note that zl^^^Zos'(i)/ni = Op(l) 
(s 7^ s') when rij —)■ cxd for i = 1, 2. Then, by eombining (IA.5I) with Theorem 2.1 
and (IA.4I) . we ean elaim the result. □ 

Proofs of Theorems 4.1 and 4.2. By eombining Theorem 2.1, Lemmas 2.1 and 
4.1, we ean elaim the results. □ 
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